Search for: All records

Training a machine learning model with data following a meaningful order, i.e., from easy to hard, has been proven to be effective in accelerating the training process and achieving better model performance. The key enabling technique is curriculum learning (CL), which has seen great success and has been deployed in areas like image and text classification. Yet, how CL affects the privacy of machine learning is unclear. Given that CL changes the way a model memorizes the training data, its influence on data privacy needs to be thoroughly evaluated. To fill this knowledge gap, we perform the first study and leverage membership inference attack (MIA) and attribute inference attack (AIA) as two vectors to quantify the privacy leakage caused by CL. Our evaluation of 9 real-world datasets with attack methods (NN-based, metric-based, label-only MIA, and NN-based AIA) revealed new insights about CL. First, MIA becomes slightly more effective when CL is applied, but the impact is much more prominent to a subset of training samples ranked as difficult. Second, a model trained under CL is less vulnerable under AIA, compared to MIA. Third, the existing defense techniques like MemGuard and MixupMMD are not effective under CL. Finally, based on our insights into CL, we propose a new MIA, termed Diff-Cali, which exploits the difficulty scores for result calibration and is demonstrated to be effective against all CL methods and the normal training method. With this study, we hope to draw the community's attention to the unintended privacy risks of emerging machine-learning techniques and develop new attack benchmarks and defense solutions. 

Secure aggregation is motivated by federated learning (FL) where a cloud server aims to compute an averaged model (i.e., weights of deep neural networks) from the locally-trained models of numerous clients, while adhering to data security requirements. Hierarchical secure aggregation (HSA) studies secure aggregation of user inputs (an abstraction of the local models) in a three-layer network with clustered users connected to the server through an intermediate layer of relays. In HSA, in addition to the conventional server security, relay security is also imposed so that the relays remain oblivious to the inputs. However, existing studies on HSA have assumed that each user is associated with only one relay, which prevents coding opportunities across inter-cluster users to achieve efficient communication and key generation. In this paper, we consider HSA with a commonly used cyclic association pattern where each user is connected to B relays in a cyclic manner. We aim to determine the best communication and security key rates in such a multi-association network. We show that when B≤K−1 (K is the total number of users), to securely compute one symbol of the desired sum of inputs, each user needs to send at least R∗X=1 symbol to the associated relays, each relay needs to send at least R∗Y=1/B symbols to the server, each user needs to hold at least R∗Z=1/B secret key symbols, and all users need to collectively hold at least R∗ZΣ=max{1,K/B−1} independent key symbols. This reveals a fundamental trade-off between the association number B and the communication and key rates. When B=K, we present a scheme that achieves the optimal communication and source key rates, along with a nearoptimal individual key rate.